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Abstract. We apply statistical physics to study the task of resource allocation in random networks with 
limited bandwidths along the transportation links. The mean-field approach is applicable when the connec- 
tivity is sufficiently high. It allows us to derive the resource shortage of a node as a well-defined function of 
its capacity. For networks with uniformly high connectivity, an efficient profile of the allocated resources is 
obtained, which exhibits features similar to the Maxwell construction. These results have good agreements 
with simulations, where nodes self-organize to balance their shortages, forming extensive clusters of nodes 
interconnected by unsaturated links. The deviations from the mean-field analyses show that nodes are 
likely to be rich in the locality of gifted neighbors. In scale-free networks, hubs make sacrifice for enhanced 
balancing of nodes with low connectivity. 



PACS. 02.50.-r Probability theory, stochastic processes, and statistics 
cations of physics 



! .20.-a Interdisciplinary appli- 



1 Introduction 



Analytical techniques developed in statistical physics have 
been widely employed in the analysis of complex systems 
in a wide variety of fields, such as neural networks [112] . 
econophysical models [3] , and error-correcting codes [214] . 
Recently, a statistical physics perspective was successfully 
applied to the problem of resource allocation on sparse 
random networks 5 6 7J . Resource allocation is a well known 
network problem in the areas of computer science and op- 
erations management [8 9|. It is relevant to applications 
such as load balancing in computer networks, reducing 
Internet traffic congestion, and streamlining network flow 
of commodities [10111] . 

In networks with finite bandwidths, the problem of 
resource allocation was studied in [12]. We derived an 
algorithm which enable us to find the optimal solutions 
without the need of a global optimizer. The mean-field 
approach was applicable when the connectivity is suffi- 
ciently high. It allows us to derive the resource shortage 
of a node as a well-defined function of its capacity, which 
corresponds to the optimized and initial resource relation. 
For networks with uniformly high connectivity we derived 
the profile of the allocated resources which exhibits fea- 
tures similar to the Maxwell construction. We generalized 
the analysis to networks with arbitrary connectivity and 
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compared the modified Maxwell construction with numer- 
ical solutions. 

In this paper, we focus on the self-organization of nodes 
in achieving a balanced environment. The analytical re- 
sults are compared with simulations, where nodes self- 
organize to balance their shortages. After defining the 
model in Section [2] we introduce the chemical potentials 
in Section [3J In Section 2] we review the theory of the 
Maxwell construction, which forms the basis for predict- 
ing the existence of clusters of balanced nodes. The emer- 
gence of balanced nodes correspond to the success in the 
uniform allocation of resources. We compare in Section 
5.11 the statistics of saturated and unsaturated links, and 
show the existence of extensive balanced clusters. The de- 
viations of the simulation results from the mean-field anal- 
yses show the dependence of final state on the locality of 
gifted and ungifted clusters in Section f572l We compare in 
Section [6] the fraction of balanced nodes in scale- free and 
regular networks and examine the role of hubs in resource 
allocation. 



2 The Model 

We consider a network with TV nodes, labelled i = 1, . . . , N. 
Each node i is randomly connected to c other nodes. The 
connectivity matrix is given by Aij = 1,0 for connected 
and unconnected node pairs respectively. Each node i has 
a capacity Ai randomly drawn from a distribution p(Ai). 
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Positive and negative values of Ai correspond to supply 
and demand of resources respectively. The task of resource 
allocation involves transporting resources between nodes 
such that the demands of the nodes can be satisfied to 
the largest extent. Hence we assign = —yji to be the 
current drawn from node j to i, aiming at reducing the 
shortage £j of node i defined by 



with 



6 



-At 



AjjUij , o 



The magnitudes of the currents are bounded by the band- 
width W, i.e., \yij\ < W. 

To minimize the shortage of resources after their al- 
location, we include in the total cost both the shortage 
cost and the transportation cost. Hence, the general cost 
function of the system can be written as 

E = /,' ^, I,, o.: //,..:• • ^o:.l,. {//,,, I,, = 1}). (2) 

(»i) ' 

The summation (ij) corresponds to summation over all 
node pairs, and Ai is a quenched variable defined on node 
i. 

In the present model of resource allocation, the first 
and second terms correspond to the transportation and 
shortage costs respectively. The parameter R corresponds 
to the resistance on the currents, and Ai is the capacity 
of node i. The transportation cost <j)(jjij) can be a general 
even function of j/y. In this paper, we consider <p and ip 
to be concave functions of their arguments, that is, <fi'(y) 
and are non-decreasing functions. Specifically, we 

have the quadratic transportation cost <fi(y) = y 2 /2, and 
the quadratic shortage cost ip(Ai, {yij\Aij = 1}) = £f/2. 



3 The Chemical Potentials and the Final 
Resources 

The optimization problem can be written as the minimiza- 
tion of Eq. © in the space of ytj and subject to the 
constraints 
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(3) 



and the constraints on the bandwidths of the links \yij \ < 
W . Introducing Lagrange multipliers to the above inequal- 
ity constraints with the Kuhn- Tucker condition, the func- 
tion to be minimized becomes 
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Vij)+lij{W + Vij) 



(4) 



Y(x) — max< —W, min 



W, 



R 



(G) 



The Lagrange multiplier \ii is referred to as the chemical 
'potential of node i, and <// is the derivative of <f> with re- 
spect to its argument. The function Y(fij — /ij) relates the 
potential difference between nodes i and j to the current 
driven from node j to i. For the quadratic cost, it consists 
of a linear segment between fij — pn = ±WR reminiscent 
of Ohm's law in electric circuits. Beyond this range, y is 
bounded above and below by ±W respectively. Thus, ob- 
taining the optimized configuration of currents y^ among 
the nodes is equivalent to finding the corresponding set of 
chemical potentials fii, from which the optimized j/ij's are 
then derived from Y([ij — fXi). This implies that we can 
consider the original optimization problem in the space of 
chemical potentials. 

The optimal currents are given by Eq. (0 in terms of 
the chemical potentials which, from Eqs. (Q]) and |4]), 
are related to their neighbors via 





I': = { K\0) 



for h-\0) > 0, 



for 



V(o) < ^ _1 (o) < o, 

-1/ 



^ ^0) for h-'(0) < -V'(0), 

where hi(x) and gi{x) are given by 



(7) 



hi{x) = -Ai - } AijY(/j,j - x), 
j 

Qi{x) = 1p' o hi(x) + X, 



(8) 



with function Y again given Eq. hiix) is the shortage 
of resource at node i when fii takes the value x. ip 1 o hi(x) 
is then the corresponding dissatisfaction cost per unit re- 
source of node j. For the quadratic shortage cost consid- 
ered in this paper, the frictionless condition ^f'(O) = is 
satisfied. Equation is then simplified to 



I'i 



min (0, -hi{m)) . 



(9) 



Hence we can interpret fii as the final shortage of resources 
after optimization. When /ij < 0, — /tj becomes the final 
resources allocated to node i. Equation ([9|) provides a sim- 
ple local iteration algorithm for the optimization problem 
in which the optimal currents can be evaluated from the 
potential differences of neighboring nodes. 

An alternative algorithm can be obtained by adopting 
message-passing approaches, which have been successful 
in problems such as error-correcting codes |I4j and prob- 
abilistic inference [15 . We refer the interested readers to 
[12j for a comprehensive derivation of the messages. 



4 The Resource Distribution Profile 



where /ij < 0, a, < 0, < and 7^ 
with respect to y^, one obtains 

Vij = Yifij - fii) 



< 0. Optimizing L 4.1 The High Connectivity Limit 

We consider the case that the bandwidth of individual 
(5) links scales as W/c when the connectivity increases, where 
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W is a constant. Thus the total bandwidth W available ^ 
to an individual node remains a constant. 

We start by writing the chemical potentials using Eq. ([9]), 



(b) 



Pi 



N 



i=i 



(10) 



In the high connectivity limit, the interaction of a node 
with all its connected neighbors become self-averaging, 
making it a function singly dependent on its own chemical 
potential, namely, 



iV 



AjjY(fij - pi) w cM(pi). 



(11) 



Physically, the function M(/i) corresponds to the average 
interaction of a node with its neighbors when its chemical 
potential is p, facilitating a mean-field approach. Thus, 
we can write Eq. (fTU|) as 

p = min[yl + cM (p), 0] , (12) 

where p is now a function of A, and 

/>oo 

M(fn) = / dAp(A)Y(p(A) - (13) 



where we have written the chemical potential of the neigh- 
bors as p(A), assuming that they are well-defined func- 
tions of their capacities A. 

To explicitly derive M(p), we take advantage of the 
fact that the rescaled bandwidth, W/c vanishes in the high 
connectivity limit, so that the current function Y(pj — pi) 
is effectively a sign function, corresponding to saturated 
links. (This approximation is not fully valid and will be 
further refined in subsequent discussions.) Thus, we ap- 
proximate 



M(ju 



W 

c 



dAp(A)sga[p(A) - p t 



(14) 



Assuming that p(A) is a monotonic function of A, and 
for Gaussian distribution of capacities, p(A) is explicitly 
given by 



/' : 



(15) 



This equation relates the chemical potential of a node, 
i.e. the shortage after resource allocation, to its initial re- 
source before. It tells us that resource allocation through 
a large number of links results in a well-defined function 
relating the two quantities. 

Eq. (fTSf gives a well-defined function p(A) as long 
as W < y/ir/2. However, when W > \JtF/2, turning 
points exists in p(A) as shown in Fig. QJa). This creates a 
thermodynamically unstable scenario, since in the region 
of p(A) with negative slope, nodes with lower capacities 
have higher chemical potentials than their neighbors with 
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Fig. 1. The Maxwell construction on p(A). 



higher capacities. Mathematically, the non-monotonicity 
of p(A) means that sgn[p(A) — pi] and sgn(yl — Ai) are no 
longer necessarily equal, and Eq. (|15p is no longer valid. 

Nevertheless, Eq. (fT0|) permits another solution of con- 
stant /i in a range of A. Hence, we propose that the unsta- 
ble region of p(A) should be replaced by a range of con- 
stant p as shown in Fig. QJb) analogous to the Maxwell 
construction in thermodynamics. Nodes within this range 
of constant p have the same amount of final resources, 
and is the consequence of the ability of the optimization 
process to balance the resources. They are referred to as 
the balanced nodes. 

In the high connectivity limit, resources are so effi- 
ciently allocated that the resources of the rich nodes are 
maximally allocated to the poor nodes. By considering 
the conservation of resources, and letting (A < ,p a ) and 
(A >: p a ) be the end points of the Maxwell construction as 
shown in Fig. |Tf b) , we have proved in [12] that 



li / dAp(A) 



dAp(A)n(A), 



(16) 



which implies that the value of \i should be chosen such 
that the areas A and B in Fig.QJb), weighted by the dis- 
tribution p(A), should be equal. 

For capacity distributions p(A) symmetric with respect 
to (A), we have fi = (A) = (A< +A > )/2. As a result, the 
function p.(A) is given by 

' (A) for A < < A < yl>, 

fW = { , . , (17) 



A - PFerf 



( A-(A) \ 



otherwise, 



where as A < and A > are respectively given by the lesser 
and greater roots of the equation x = (A) + Weri [(x — 
(A))/V2}. 

We compare the analytical result of p(A) in Eq. (jTTJ) 
with simulations in Fig. [2] For W > y/n/2, data points 
[A, p) of individual nodes from network simulations follow 
the analytical result of p(A), giving an almost perfect over- 
lap of data. The presence of the balanced nodes with effec- 
tively constant chemical potentials is obvious and essen- 
tial to explain the behavior of the majority of data points 
from simulations. On the other hand, for W < Wtt/2, the 
analytical p(A) shows no turning point as shown in the 



4 



C. H. Yeung and K. Y. M. Wong: Self-organization of Balanced Nodes 




-6 -4 -2 2 4 



V i i i i i i a l 

-6 -4-2 2 

A 



Fig. 2. The simulation results of fx(A) for N = 10000, c = 
15, R = 0.1, (A) = -1 and W = 3 with 70000 data points, 
compared with theoretical prediction. Inset: The corresponding 
results for W — 1.2. 




Fig. 3. Simulation results of (A, p) for TV = 2000, W = 3, 
c = 10 and (A) = —5 shown with an expanded vertical scale, 
as compared with the theoretical predictions of (i(A) from 
Eq. fI7)l. Eq. dTHJ and the numerical solution of Eq. J2D. In- 
set: same data set and theoretical predictions in the normal 
vertical scale. 



inset of Fig. O Despite the scattering of data points, they 
generally follow the trend of the theoretical p(A). 



4.2 The Cases with General Connectivity 

Our analysis can be generalized to the case of large but 
finite connectivity, where the approximation in Eq. Q14p 
is not fully valid. This modifies the chemical potentials of 
the balanced nodes, for which Eq. (fT4"]) has to be replaced 

by 



M(/i) 



dAp(A) - 



W r r °° ' ! 

c 
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dAp(A) 



R 



(18) 



We introduce an ansatz of a linear relationship between p 
and A for the balanced nodes, namely, 



ji = mA + b. 



(19) 



After direct substitution of Eq. (fT9]l into M(p) given by 
Eq. ([18]). we get the self-consistent equations for m and b, 

R 



b = 



R + c erf 



V2 



R 



erf 



(^) 



(A). 



(20) 



Thus, the Maxwell construction has a non-zero slope when 
the connectivity is finite. 

We remark that the approximation in Eq. (jT5J) assumes 
that the potential differences of the balanced nodes lie in 



the range of 2RW/c, so that their connecting links remain 
unsaturated. Note that the end points of the Maxwell con- 
struction have chemical potentials (A) ± RW/c respec- 
tively, rendering the approximation in Eq. (|18j) exact at 
one special point, namely, the central point of the Maxwell 
construction. Hence, this approximation works well in the 
central region of the Maxwell construction, while devia- 
tions are expected near the end points. 

We compare Eq. (fT8]) with the n(A) given by the nu- 
merical solution of the integral equation 

/oo 
dAp{A)Y[n(A) - fi(Ai)}. (21) 

Since iterating this equation may lead to an oscillating so- 
lution of fi(A), we solve it by gradient descent. The results 
are shown in Fig. [3] in an enlarged scale of /i. As expected, 
Eq. (fTS]) works well around fi = (A) and show small devi- 
ations at the end points of the Maxwell construction. In 
comparison with simulations, data porints are scattered 
from the theoretical predictions, but generally follow the 
slanted path of p{A) rather than the horizontal path as 
predicted by Eq. (TT7|) . We will explain the scattering of 
data points in the next section. As shown in the inset of 
Fig. [3l the differences among the different approaches are 
not obvious unless the scale of fi is expanded. We thus 
conclude that both the horizontal and slanted Maxwell 
constructions are good approximations of fJ,(A) and cap- 
ture the general trend of the simulation data. 

Remarkably, as evident from Eq. (|2"0)) , even with con- 
stant available bandwidth W, increasing connectivity causes 
m to decrease, and hence sharpens the chemical poten- 
tial distribution. The narrower distributions correspond 
to higher efficiency in resource allocation. It leads us to 
realize the potential benefits of increasing connectivity in 
network optimization even for a given constant total band- 
width connecting a node. 
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Fig. 4. Analytical predictions of /bai as compared with simu- 
lation results for networks of N = 10000, W = 3, R = 0.1 and 

(A) = -5. 



5 The Self-organization of the Balanced 
Nodes 

The fraction /b a i of balanced nodes is given by the equa- 
tion 



bai 



erf 



Wfbal 

V2 



(22) 



Note that /b a i bas the same dependence on W for all 
negative (A) . Figure [4] shows that when the total band- 
width W increases beyond yJir/2, the analytical fraction 
of balanced nodes increases, reflecting the more efficient 
resource allocation brought by the convenience of increased 
band widths. When W becomes very large, a uniform chem- 
ical potential of (A) networkwide is recovered, converging 
to the case of non- vanishing bandwidths 12J. 

We measure /bai m simulations as follows. As only 
finite connectivity can be implemented, we define node 
i to be balanced when its chemical potential falls into 
the slanted range of the Maxwell construction, i.e. (A) — 
RW/c < fa < (A) + RW/c. The simulation results are 
compared with the analytical results in Fig. [4j Deviations 
are found at intermediate values of W, which may be ex- 
pained by the scattering of simulated data points. Nev- 
ertheless, increass in /b a i are observed at W » \A"/2, 
corresponding to the emergence of balanced nodes in sim- 
ulations. 



5.1 The Extensive Clusters of Balanced Nodes 

In random networks, balanced nodes are found in clus- 
ters interconnected by an extensive fraction of unsaturated 
links. The clusters connect most balanced nodes and span 
the whole network when a large fraction of balanced node 
is found. The unsaturated links in the clusters provide 
the freedom to fine tune their currents so that the short- 
ages among the nodes are uniform. These features are the 
natural consequences of optimization, in which nodes self- 
organize to balance their shortages. 
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Fig. 6. The schematic relation of the resources of a node 
(a) before and (b) after optimization. The division of nodes 
into gifted rich, gifted poor, ungifted rich and ungifted poor is 
shown in (b). 



To examine the clustering of balanced nodes, we show 
in Fig.[5](a) and (b) respectively the distributions P(fa, faj \ \yij \ < 
W) and P(fa , faj \ \ — W) , which correspond to the joint 
probability distributions of the final resources of termi- 
nal nodes i and j for link (ij), given the link is unsatu- 
rated and saturated. As shown in Fig. [5] (a), unsaturated 
links connect nodes with fa w fij. A prominent peak is 
found around fa w faj w (A), corresponding to unsatu- 
rated linkages between the balanced nodes. On the other 
hand, P(fa, fij\\yij\ = W) shows non-zero probabilities in 
regions other than fa m fij. Peaks are observed, which are 
similar to a Gaussian distribution with the central slice 
removed. Non-zero probabilities are observed along the 
axes fa « (yl) and /ij f=a (A), corresponding to saturated 
linkages between balanced and unbalanced nodes. These 
results support the existence of balanced clusters inter- 
connected by an extensive fraction of unsaturated links. 



5.2 The Neighborhood of Rich and Poor Nodes 

To understand the scattering of data points of (fa A) from 
the mean-field predictions, we identify the role of the nodes 
in resource allocation according to their capacities. Nodes 
with capacities greater and less than (yl) are respectively 
referred to as the gifted and ungifted nodes. Figure [6] shows 
the schematic relation of the resources of a node before 
and after optimization. Before optimization, the resource 
of a node is equal to its capacity. After optimization, the 
resource of a node is equal to /i. Gifted nodes have their 
resources reduced after donating them, and ungifted nodes 
have their resources increased after receiving them. fa[A) 
is then described by the Maxwell construction as derived 
in the mean-field analysis. 

As finite connectivity is implemented in simulations, 
the neighborhood of a node deviates from the mean-field 
descriptions which results in scattering of data points (fa A). 
To examine the effect of the locality of nodes in relation 
with their final shortage, we define rich and poor nodes to 
be nodes with final resources higher and lower than the 
mean- field predictions. As shown in Fig. [5Jb), the nodes 
are thus categorized into gifted rich, gifted poor, ungifted 
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Fig. 5. The distributions (a) P([J.i, [ij \ \yij\ < W) and (b) P([H, /J-j\\yij\ = W), corresponding to the joint probability distri- 
butions of the final resources of terminal nodes i and j for link (ij), given the link is unsaturated and saturated respectively. 
Parameters: N = 10000, W = 3, R = 0.1 and (A) = -5. 
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Fig. 7. The fraction of gifted rich, gifted poor, ungifted rich 
and ungifted poor nodes among the nearest neighbors of a node 
in a particular category. Parameters: N = 10000, W = 3, R = 
0.1, (A) = —5 and 5 samples. 



rich and ungifted poor, in accordance to the scattering of 
the capacity-shortage relations i.e. (/x, A) of the node. As 
an example, gifted rich nodes are nodes with initial ca- 
pacity higher than (A) and final resources higher than the 
mean-field predictions, as shown in Fig.[S^b). 

We examine the neighborhood of a node in Fig. [7] by 
measuring the fractions of gifted rich, gifted poor, ungifted 
rich and ungifted poor nodes among its nearest neighbors. 
As compared with the random case, a high ratio of gifted 
rich node is found surrounding a gifted rich node. In other 
words, gifted nodes are more likely to be rich in the neigh- 
borhood of gifted nodes, forming a cluster of rich nodes 
after allocation. Physically, the effective average capacity 
is higher than (A) in the locality of gifted clusters, leading 
to higher resources than the mean-field predictions. The 
converse is true for ungifted poor nodes, which results in 
lower resources in the locality of ungifted clusters. On the 
other hand, gifted nodes are more likely to be poor if they 



-0.5 



Regular networks 
x Scale-free network 
— Theoretical 



-1.5 




-2.5 



Fig. 8. Simulation results of (A, /x) for networks of N — 2x 10 , 
W = 3, R = 0.1 and {A) = —1 with uniform connectivity 
of c = 3 and scale-free network of P(c) ~ c -3 with c > 3, 
each with 2500 data points, as compared with the theoretical 
predictions of Eq. I|20p . Inset: the /bai as a function of c for 
nodes in scale-free and regular networks with N=10000 and 50 
samples. 



are in the neighborhood of ungifted clusters, as shown by 
the statistics of the neighbors of gifted poor nodes, and 
vice versa. We thus conclude that the final state of a node 
is highly dependent on its locality, which results in the 
scattering of the simulated data points of (//, A) around 
the prediction of the mean-field analyses. 



6 Balanced Nodes in Scale-Free Networks 



We have examined the features of balanced nodes in reg- 
ular networks . However, recent studies of complex net- 
works show that many realistic communication networks 
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have highly heterogeneous structure, and the connectiv- 
ity distribution obeys a power law |16j . These networks, 
commonly known as scale-free networks, are characterized 
by the presence of hubs, which are nodes with very high 
connectivities, and are found to modify the network be- 
havior significantly. Hence, it is interesting to study the 
allocation of resources and the features of balanced nodes 
in scale-free networks. We define the bandwidth of the 
link (ij) to be Wij = W / max(ci, Cj), where Cj and Cj are 
the connectivity of the terminal nodes. In this case, nodes 
in scale-free network may have a smaller effective W, as 
compared with their counterpart in regular networks with 
identical connectivity. 

The simulation results are presented in Fig. [8j where 
we plot the data points of (A, /i) from nodes of c = 3 in 
scale-free networks. Despite their low connectivity, their 
capacity-shortage relations exhibit the flat distribution 
characteristic of the Maxwell construction, coinciding with 
the analytical results of the high connectivity limit. This 
shows that the presence of hubs in scale-free networks in- 
creases the global efficiency of resource allocation, leading 
to balanced shortages on nodes with low connectivity. To 
confirm this advantage of the scale-free topology, we also 
plot in the figure the data points obtained from networks 
of uniform connectivity c — 3. Evidently, the data points 
are much more scattered away from the Maxwell construc- 
tion. 

However, the enhanced balancing in scale- free networks 
are found only for nodes with low connectivity. We com- 
pare in the inset of Fig. [5J the /bai in scale-free networks 
and regular networks, for nodes with higher connectivities. 
From the firgure for c = 3, 4, a much higher /b a i is found in 
scale-free networks than their counterparts in regular net- 
works. The opposite is true for c > 5, and the differences 
increases with c. It implies that the nodes with higher 
connectivity in scale-free neworks sacrifice themselves for 
balancing the majority of nodes with low connectivity. In 
contrast, the fraction of balanced nodes increases with the 
connectivity in regular networks. This picture thus clar- 
ifies the role of hubs in resource allocation on scale-free 
networks. 



7 Conclusion 

We have applied statistical mechanics to study an opti- 
mization task of resource allocation on a network, in which 
nodes with different capacities are connected by links of fi- 
nite bandwidths. By adopting suitable cost functions, such 
as quadratic transportation and shortage costs, the model 
can be applied to the study of realistic networks. The 
mean-field approach valid in the high connectivity limit 
enables us to derive the capacity-shortage relations, and 
study the deviations from this limit for finite connecitivty. 

In particular, the study reveals interesting effects due 
to finite bandwidths. A remarkable phenomenon is found 
in networks with fixed total bandwidths per node, where 
bandwidths per link vanish in the high connectivity limit. 
For sufficiently large total bandwidths, clusters of bal- 
anced nodes self-organize to have a uniform shortage rem- 



iniscent of the Maxwell construction in thermodynamics. 
The locality of gifted and ungifted clusters respectively 
lead to the formation of rich and poor clusters. In scale- 
free networks, hubs are more likely to be unbalanced and 
make sacrifice for nodes with low connectivity to get bal- 
anced. We believe that the present analyses of balanced 
nodes lead us to better understanding of self-organization 
in resource allocation, as well as other systems. 
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